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CLUSTERING 
BACKGROUND 

This invention relates to clustering, for example, of seasonalities to 
better forecast demand for items of commerce. 

5 Clustering means grouping objects, each of which is defined by 
values of attributes, so that similar objects belong to the same 
cluster and dissimilar objects belong to different clusters. 
Clustering has applications in many fields, including medicine, 
astronomy, marketing, and finance. 

1 0 Clustering is done on the assumption that attribute values 
representing each object to be clustered are known 
deterministically with no errors. Yet, often, the values representing 
an object to be clustered are not available. Sometimes statistical 
methods are used to get estimated or average values for a given 

15 object. 

SUMMARY 

In general, in one aspect, the invention features a method that 
includes (a) receiving a set of data containing values associated 
with respective data points, the values associated with each of the 

20 data points being characterized by a distribution, (b) expressing the 
values for each of the data points in a form that includes 
information about a distribution of the values for each of the data 
points, and (c) using the distribution information in clustering the 
set of data with at least one other set of data containing values 

25 associated with data points. 
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Implementations of the invention may include one or more of the 
following features. The respective data points are related in a time- 
sequence. The data points relate to a seasonality of at least one 
item of commerce. Each of the sets of data relates to seasonalities 
5 of items of commerce. The items of commerce comprise retail 
products, the data points relate to times during a season, and the 
values associated with each of the data points correspond to 
respective ones of the retail products. The method also includes 
determining statistical measures of the variability of the values 

1 0 with respect to the data point. The data is expressed in a form that 
includes a mean of the values associated with a data point and a 
statistical measure of the distribution with respect to the mean. The 
statistical measure comprises a standard deviation. The clustering 
of data includes measuring a distance between pairs of the sets of 

1 5 data. The distance is measured based on the means and variances at 
the data points. The distribution of the values is Gaussian. The 
clustering of data includes merging the data sets belonging to a 
cluster using a weighted average. The method includes merging the 
seasonalities of the data sets belong to a cluster. 

20 In general, in another aspect, the invention features a machine- 
accessible medium that when accessed results in a machine 
performing operations that include: (a) receiving a set of data' 
containing values associated with respective data points, the values 
associated with each of the data points being characterized by a 

25 distribution, (b) expressing the values for each of the data points in 
a form that includes information about a distribution of the values 
for each of the data points, and (c) using the distribution 
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information in clustering the set of data with at least one other set 
of data containing values associated with data points. 

In general, in another aspect, the invention features a method that 
includes (a) receiving sets of data, each of the sets containing 
5 values associated with respect data points, the values associated 
with each of the data points being characterized by a distribution, 
(b) evaluating a distance function that characterizing the similarity 
or dissimilarity of at least two of the sets of data, the distance 
function including a factor based on the distributions of the values 
1 0 in the sets, and (c) using the evaluation of the distance function as 
a basis for clustering of the sets of data. 

In general, in another aspect, the invention features a method that 
includes (a) receiving data that represents seasonality time-series 
for each of a set of retail items, the data also representing error 
1 5 information associated with data values for each of a series of time 
points for each of the items, and (b) forming composite seasonality 
time-series based on respective clusters of the retail item 
seasonality time-series, the composites formed based in part on the 
error information. 

20 Other advantages and features will become apparent from the 
^Following description and ^m the claimsT 

DESCRIPTION 

(Figure 1 shows product life cycle curves. 

Figure 2 shows a seasonality curve and two sales curves. 
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Figure 3 shows a concatenation of PLC curves and an average 
curve. 

Figure 4 shows three seasonality curves. 
Figure 5 illustrates the merger of time-series. 
5 Figure 6 illustrates the merger of time-series. 

Figures 7, 8, 9, 10 and 1 1 show seasonality curves. 
Figure 12 is a flow chart.) 
Introduction 

The clustering of objects can be improved by incorporating 
1 0 information about errors, which are a measure of level of 

confidence, in data that characterize the objects to be clustered. 
Instead of relying on simple average values for attributes of the 
objects, measures of error such as standard deviation or an estimate 
of an entire distribution of the data for a value can be used to 
1 5 improve the quality of the clustering. A new distance function that 
is based on the distribution of errors in data (and may be viewed as 
a generalization of the classical Euclidian distance function) can be 
usedjn clustering.^ 

Although the following discussion focuses on the utility of the new 
20 distance function for time-series clustering, the concept of 

incorporating information about error in the distance function can 
be used in developing a distance function in any clustering 
application in which data that is clustered in other ways. 
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The problem 

We describe the problem being solved in the following new way: 

Let m objects (e.g., seasonalities) be represented by m data points 
(e.g., sales demand rates) in /^-dimensional space (e.g., 
5 representing n weeks) . Each of these m data points represents a 
Gaussian distribution of data values, which defines the average 
value of each data point and also specifies the standard error 
associated with each data point. 

We group these data points into clusters so that it is likely that data 
10 points (sales demand rates) in the same cluster are similar to (close 
to) each other, and data points in different clusters are unlikely to 
be close to each other, the likelihood being defined with respect to 
the Gaussian distributions represented by the data points. In 
contrast to other clustering techniques, two data points that differ 
1 5 significantly in their means may belong to the same cluster if they 
have high errors associated with them, and two data points that do 
not differ much in their means might belong to different clusters if 
they are well-calculated and have small errors. 

Seasonality forecasting 

20 As one example, we consider the forecasting of seasonality for 
retailers based on sales data from previous years. Seasonality is 
defined as the underlying demand for a group of similar 
merchandise items as a function of time of the year that is 
independent of external factors like changes in price, inventory, 
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and promotions. Seasonality for an item is expected to behave 
consistently from year to year. 

In the retail industry, it is important to understand the seasonal 
behaviors in the sales of items to correctly forecast demand and 
5 make appropriate business decisions with respect to each item. 
Different items may be characterized by different seasonalities for 
a year. Yet, to reduce random variability and make better demand 
forecasts, it is desirable to group items with similar seasonal 
behavior together. 

1 0 In this application, the objects to be clustered are the seasonalities 
of different groups of retail items. Effective clustering of retail 
items yields better forecasting of seasonal behavior for each of the 
retail items. Therefore, we identify a set of clusters of seasonalities 
that model most of the items sold by the retailer and relate each 

1 5 item to average seasonality of the cluster. 

We model seasonality forecasting as a time-series clustering 
problem in the presence of errors, and we discuss experimental 
results on retail industry data. We have discovered meaningful 
clusters of seasonality that have not been uncovered by clustering 
20 methods that do not use the error information. 

We assume that external factors, like price, promotions, and 
inventory, have been filtered from our data, and that the data has 
been normalized so as to compare sales of different items on the 
same scale. After normalization and filtering for external factors, 
25 the remaining demand rate of an item is determined by its Product 
Life Cycle (PLC) and seasonality. PLC is defined as the base 
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demand of an item over time in the absence of seasonality and all 
other external factors. 

Filtering product life cycle effects 

As shown in figure 1, for example, in a typical PLC 14, an item is 
5 introduced on a certain date 1 0 and removed from stores on a 
certain date 12. PLC 14 is a curve between the introduction date 
and removal date. The shape of the curve is determined by the 
duration of time an item is sold 16 and also the nature of the item. 
For example, a fashion item (right-hand curve 15) will sell out 
1 0 faster a non-fashion item (left-hand curve 14). For simplicity, we 
assume the PLC value to be zero during weeks when the item is 
not sold. 

Because sales of an item is a product of its PLC and its seasonality, 
it is not possible to determine seasonality just by looking at the 
1 5 sales data of an item. The fact that items having the same 

seasonality might have different PLCs complicates the problem. 

For example, if both the items for which PLC curves are shown in 
figure 1 follow the same seasonality 20 as shown in figure 2, then 
the sales of the two items (non-fashion and fashion) will be as 
20 shown in the two different curves 22 and 24 shown in the right- 
hand side of figure 2. Curves 22 and 24 are respectively the 
products of curves 14, 15 of figure 1 and the seasonality curve 20 
in figure 2. 

The first step is to remove as much as possible of the PLC factor 
25 from the sales data so that only the seasonality factor remains. 
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Initially, based on prior knowledge from merchants, we group 
items that are believed to follow similar seasonality over an entire 
year. As shown in figure 3, the items in the set 30 follow similar 
seasonality but may be introduced and removed at different points 
5 of time during the year. The set typically includes items having a 
variety of PLCs that differ in their shapes and durations of time. 
The weekly average of all PLCs in this set is a somewhat flat curve 
32 as shown on the right-hand side of figure 3, implying that the 
weekly average of PLCs for all items in the set can be assumed to 
10 be constant. 

This implies that averaging of weekly filtered demand rates of all 
items in the set would nullify the effect of PLCs and would 
correspond to the common value of seasonality indices for the 
items in the set to within a constant factor. The constant factor can 
1 5 be removed by appropriate scaling of weekly sales averages. 
Although the average values obtained above give a reasonably 
good estimate of seasonality, they will have errors associated with 
them depending on how many items were used to estimate 
seasonality and also on how well spread are their PLCs. 

20 Error information 

Let a j be the variance of all the filtered demand rates for a set 

(e.g., one of the sets described above) of retail items in week j. To 
be clear, we are capturing the variance of the data values that 
represent the units of sales of the various items for a given week j 
25 as found in, say, one year of historical sales data. If there are a total 
of m items in this set, then an estimate of Sj, which is the standard 
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error in the estimation of the seasonality index for week j y is given 
by the following equation. 

s ' - 2 (i) 

By seasonality index for week j, we mean the factor that represents 
5 the seasonality effect for all m items in the set relative to the 
baseline demand value. So, if the baseline demand is 20 units in 
week/ and the seasonality index in week j is 1.3 then we expect 
the sales data in week j to be 26 units. 

The above procedure provides a large number of seasonal indices, 
10 one for each set of retail items of merchandise, along with 

estimates of associated errors. We group these seasonal indices 
into a few clusters based on the average values and errors as 
calculated above, thus associating each item with one of these 
clusters of seasonal indices. The seasonality index of a cluster is 
1 5 defined as a weighted average of seasonalities present in the cluster 
as shown by equation 3 below. The cluster seasonality obtained in 
this way is much more robust because of the large number of 
seasonalities used to estimate it. 

To summarize the discussion to this point, based on prior 
20 knowledge-aboutwhich items; x>f the thousands of items that 
appear in a merchant's hierarchy of items, follow similar 
seasonality, we group items into sets of items that have similar 
seasonality. We estimate the seasonality for each of these sets and 
also estimate the error associated with our estimate of seasonality. 
25 This provides us with a large number of seasonalities along with 
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associated errors. We cluster these seasonalities into a few clusters, 
then calculate the average seasonality of each cluster. The final 
seasonality of an item is the seasonality of the cluster to which it 
belongs. 

5 Generating seasonality forecasts by clustering without 

incorporating the errors information would disregard the fact that 
we do not know how much confidence we have in each 
seasonality. If we have high confidence in the estimate of a 
seasonality, then we would like to be more careful in assigning it 

10 to a cluster. On the other hand, if we have little confidence in the 
estimate of a seasonality, then its membership in a cluster does not 
have high significance in the cumulative average seasonality of the 
cluster. Errors or the associated probability distributions capture 
the confidence level of each seasonality and can be used 

1 5 intelligently to discover better clusters in the case of stochastic 
data. 

Representation of time-series 

Time-series are widely used in representing data in business, 
medical, engineering, and social sciences databases. Time-series 
20 data differs from other data representations in the sense that data 
points in a time-series are represented by a sequence typically 
measured at equal time intervals. 

Stochastic data can be represented by a time-series in a way that 
models errors associated with data. 
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A time-series of stochastic data sampled at k weeks is represented 
by a sequence of k values. In our application, where we assume, 
for example, that the k samples are independent of each other and 
are each distributed according to one-dimensional Gaussian 
5 distribution, we represent a time-series A as: 

A= {U^,,wJ,(// 2 ,5 2 ,w 2 ),...,^,^,wJ} where the 
stochastic data of the f h sample of A is normally distributed with 
mean //, and standard deviation s f . w { is a weight function to give 
relative importance to each sample of the time-series. Weights are 

1 0 chosen such that * Wf ) = k so as to express time-series on the 
same scale. This normalization is important in the following sense. 
First of all, these weights reflect the relative, not absolute, 
importance of each sampled value. Secondly, the normalization 
converts the data into a unit scale thereby comparing differences in 

15 the shapes of time-series and not in the actual values. In figure 4, 
for example, the normalization will facilitate putting the time- 
series A and the time-series B in the same cluster and the time- 
series C in a separate cluster. 

Although one can experiment with different weights for different 
20 sample values of a time-series, for simplicity, we assume that all 
the k sample values of a time-series have equal weight. Henceforth, 
we will work with the following compact representation of time- 
series. A = lp\ iS \\(p\,s' 2 }.~,(p\,s' k )}, where ju\=^*w and 

k 

s'i = si * w , for i = 1,2,..., k where w = . 
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In the case of seasonality forecasting, we may have k equal to 52 
corresponding to 52 weeks in a year, and //, be the estimate of the 
seasonality index for f* week. s t represents the standard error in the 
estimated value of fi t . 

5 We have assumed that sales data that are used to estimate means 
and errors of the respective k sample values of a time-series come 
from independent distributions. Although we might observe some 
level of independence, complete independence is not possible in 
real data. Especially in time-series data, one expects a positive 

10 correlation in consecutive sample values. Therefore, while 

incorporating the concept of dependence can be difficult, it can 
improve the test statistic for distance and subsequently give a more 
accurate measure of the distance function. In the case of 
seasonality forecasting, we deal with seasonality values that are 

1 5 obtained by taking the average of sales data of items having 
different PLCs, These PLCs are pretty much random for a large 
sample of data and therefore averaging over these random PLCs 
may dampen the effect of dependency among different samples. 
This implies that dependency is not a serious issue in our 

20 application, a proposition that is also observed experimentally. 

Distance function 

As in other clustering methods, we make a basic assumption that 
the relationship among pairs of seasonalities in a set of n 
seasonalities is described by an nXn matrix containing a measure 
25 of dissimilarity between the i th and the f h seasonalities. In 

clustering parlance, the measure of dissimilarity is referred to as a 
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distance function between the pair of seasonalities. Various 
distance functions have been considered for the case of 
deterministic data. We have developed a probability-based distance 
function in the case of multidimensional stochastic data. 

5 Consider two estimated seasonality indices 

a j =(^P^J(/ / y2^yJ^W^y*) .-4/ and 4 are the estimated 
time-series of two seasonalities based on historically observed 
sales for corresponding sets of items. Let the corresponding true 
10 seasonalities be {ji n ,ji n9 ...,/J ik } and \p^Ji j29 .^Jij k .This means 

that the jj 's are the observed means that are associated with the 
true means of JJ 's. 

We define similarity between two seasonalities (time-series) as the 
probability that the two seasonalities might be the same. Two 

15 seasonalities are considered the same if the corresponding fx 9 s are 
close with high significance with respect to the associated errors. 
In other words, if we define the null hypothesis Ho^sAf-Aj then 
similarity between A f and Aj is the significance level of this 
hypothesis. Here, A t = Aj means corresponding Ji u = Jiji for 

20 / = 1 k. The distance or dissimilarity dy between A\ and Aj is 
defined as (1 - similarity). In other words, dy is the probability of 
rejecting the above hypothesis. This distance function satisfies the 
following desirable properties. 

dist(A,B) = dist(B,A) 
25 dist(A t B) >_0 
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dist(A,A) = 0 
dist(A,B)=0=>A=B 

*i 

where is the pooled variance for week / defined as 

5 Under a Gaussian assumption, the above statistic follows a Chi- 
Square distribution with k-\ degrees of freedom. Therefore, the 
distance d y which is the significance level of rejecting the above 
hypothesis is given by the following equation. 




d^ChiSqr^PDF £ \2-S£\ ^-1 



(2) 



We use this distance between pairs of seasonality indices as the 
basis for clustering them by putting pairs that have low distances in 
the same cluster and pairs that have high distances in different 
clusters. We use a confidence interval (e.g., 90%) to find a 
threshold distance. If the distance between two seasonalities is less 
1 5 than the threshold distance value, we put them in the same cluster. 
Merging time-series 

Here we define a 'merge 1 operation to combine information from a 
set of time-series and produce a new time-series that is a 
compromise between all the time-series used to produce it. In 
20 seasonality forecasting, the time-series are sets of seasonalities and 
the new time-series represents the weighted average seasonality for 
a cluster of seasonalities. The shape of the resulting time-series 
depends not only on the sample values of individual time-series but 
also on errors associated with individual time-series. 
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Given r time-series A t = \p n , sn\ {j* n » J^jl— » ip ik , si k \ i = l,2,.~r 
then the resulting time-series C = {(ju l ,s ] \(ju 29 s 2 ),... 9 {{i k9 s k )} is 
given by 

/-l 2 

Mj = Y" y = U2 A (3) 

S(, 

5 ■ 1 . ;=1,2,...,* (4) 



As shown in the example of figure 5, consider two time-series 40 
and 42 in which the curves 44, 46 represent the average values at 
each of 26 samples of time represented along the horizontal axis. 
The vertical line 48 for each of the samples represents the error 

1 0 associated with the sample at that time. The time-series 50 is the 
resulting sequence when the time-series 40 and 42 are merged in 
the manner discussed above. As can be seen, portions of each of 
the time-series 40 and 42 for which the errors are relatively smaller 
than in the other time-series serve more prominently in the merged 

1 5 time series 50. Therefore, if the series 50 represented a merged 
seasonality of two merchandise items as part of a cluster of items, 
a retailer could use the series 50 as a more accurate prediction of 
seasonality of the items that make up the cluster than would have 
been represented by clustered seasonalities that were merged 

20 without the benefit of the error information. This helps the retailer 
make better demand forecast as shown by experimental results. 
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Experimental Results 

We generated data sets as described below. First we generated ten 
different kinds of PLCs from a Weibull distribution with different 
parameters. These PLCs differ in their peaks and shapes depending 
5 on the parameters used in the distribution, as shown in figure 6. 
The PLC data is randomly generated by choosing one of these 10 
PLCs with equal probability and a uniformly distributed starting 
time over a period of one year. 

Then we considered three different seasonalities corresponding to 
10 Christmas seasonality, summer seasonality, and winter seasonality 
respectively, as shown in figure 7. We generated sales data by 
multiplying the randomly generated PLC data with one of the three 
seasonalities. When we considered twelve instances, each instance 
produced by generating 25-35 PLCs, multiplying them by one of 
1 5 the above seasonalities and averaging weekly sales to obtain an 
estimate of corresponding seasonality, we obtained the values of 
seasonalities with associated estimates of errors as shown in figure 
8. 

As shown in figure 8, some of the seasonalities do not correspond 
20 to any of the original seasonalities and each has large errors. We 
ran the clustering method as described above and we got three 
clusters with cluster centers as shown in figure 9, where cluster 
centers are obtained by averaging of all PLCs in the same cluster 
according to equations 3 and 4. The resulting seasonalities match 
25 the original seasonalities of figure 7 well as shown in figure 9. We 
compared our result with standard hierarchical clustering that did 
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not consider the information about errors. The number of 
misclassifications were higher when we used hierarchical 
clustering with standard Euclidean distance without accounting for 
errors; as shown in figure 10. 

5 Actual Data Results 

Figure 1 1 shows an example based on actual retail sales data. 
Each of the seven time-series in this figure represents a seasonality 
that is obtained from a group of items that are known to follow 
similar seasonality. The clustering of the seven sets provides five 
1 0 clusters as shown in the figure. 

The expressing of data in a way that incorporates error 
information, the expressing of a distance function based on the 
error information, the clustering of data sets using the distance 
function, the merging of clusters of data sets, the specific 
1 5 applications of the techniques to time-series data, including 
seasonality time-series for retail items, and other techniques 
described above, can be done using computers or machines 
controlled by software instructions. 

For example, as shown in figure 12, the software would be stored 
20 in memory or on a medium or made available electronically, and 
would enable the computer or machine to perform the following 
sequence, for example, in the context of a retailer making pricing 
or inventory decisions with respect to retail items. After filtering 
and normalizing the historical time-sequence sales data for a 
25 number of items, in step 80, the data would be grouped and 

processed in step 82 to reduce the effect of the product life cycle 
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factor. This would include grouping items known to have similar 
seasonality, scaling, and determining variance and standard errors 
with respect to each group. The seasonality indices for each of the 
groups would then be expressed in a representation (step 84) that 
5 captures both (a) the mean values of seasonality for a set of items 
for each period and (b) the statistical or error information that 
represents the confidence level with respect to the mean values of 
seasonality. The software would analyze the various seasonality 
time-series using the distance functions (step 86) to cluster the 

1 0 time-series, and then would use the clustered time-series as part of 
the basis for decision-making (step 88). Additional information 
about the collection of such information and the making of such 
decisions is found in United States patent applications 09/263,979, 
filed March 5, 1999, 09/826,378, filed April 4, 2001, and 

1 5 09/900,706, filed July 6, 2001 , all incorporated by reference here. 

Other implementations are within the scope of the following 
claims. 
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CLAIMS 

11. A method comprising 

2 receiving a set of data containing values associated with 

3 respective data points, the values associated with each of the data 

4 points being characterized by a distribution, 

5 expressing the values for each of the data points in a form 

6 that includes information about the distribution of the values for 

7 each of the data points, and 

8 using the distribution information in clustering the set of 

9 data with at least one other set of data containing values associated 
10 with data points. 

1 2. The method of claim 1 in which the respective data points 

2 are related in a time-sequence. 

1 3. The method of claim 1 in which the data points relate to a 

2 seasonality of at least one item of commerce. 

14. The method of claim 1 in which each of the sets of data 
2 relates to seasonalities of items of commerce. 

15. The method of claim 4 in which the items of commerce 

2 comprise retail products, the data points relate to times during a 

3 season, and the values associated with each of the data points 

4 correspond to respective ones of the retail products. 
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1 6. The method of claim 5 also including determining 

2 statistical measures of the variability of the values with respect to 

3 the data point. 

1 7, The method of claim 1 in which the data is expressed in a 

2 form that includes a mean of the values associated with a data 

3 point and a statistical measure of the distribution with respect to 

4 the mean. 

1 8. The method of claim 7 in which the statistical measure 

2 comprises a standard deviation. 

1 9. The method of claim 1 in which the clustering of data 

2 includes measuring a distance between pairs of the sets of data. 

1 10. The method of claim 9 in which the distance is measured 

2 based on the means and variances at the data points. 

1 11. The method of claim 1 0 in which the distance is measured 

2 based on dij - ChiSqr _ PDF 





r 




2 \ 


PDF 


k 

I 


,*~1 




\ 


< s t J 


J 



1 12. The method of claim 1 in which the distribution of the 

2 values comprises a Gaussian distribution. 

1 13. The method of claim 1 in which the clustering of data 

2 includes merging the data sets belonging to a cluster using a 

3 weighted average. 

1 14. The method of claim 1 also including merging the 

2 seasonalities of the data sets belong to a cluster 
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1 15. A machine-accessible medium that when accessed results 

2 in a machine effecting actions comprising: 

3 receiving a set of data containing values associated with 

4 respective data points, the values associated with each of the data 

5 points being characterized by a distribution, 

6 expressing the values for each of the data points in a form 

7 that includes information about a distribution of the values for each 

8 of the data points, and 

9 using the distribution information in clustering the set of 

1 0 data with at least one other set of data containing values associated 

1 1 with data points. 

1 16. A method comprising 

2 receiving sets of data, each of the sets containing values 

3 associated with respect data points, the values associated with each 

4 of the data points being characterized by a distribution, 

5 evaluating a distance function that characterizes the 

6 similarity or dissimilarity of at least two of the sets of data, the 

7 distance function including a factor based on the distributions of 

8 the values in the sets, and 

9 using the evaluation of the distance function as a basis for 
1 0 clustering of the sets of data. 

1 17. A method comprising 

2 receiving data that represents seasonality time-series for 

3 each of a set of retail items, the data also representing error 
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4 information associated with data values for each of a series of time 

5 points for each of the items, and 

6 forming composite seasonality time-series based on 

7 respective clusters of the retail item seasonality time-series, the 

8 composites formed based in part on the error information. 
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Figure 1 
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Figure 3 
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Figure 5 
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Figure 6 




Figure 7 
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Figure 9 
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Figure 10 
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Figure 11 
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